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I.  Summary:  Objectives  and  Status  of  Effort 

In  this  report  we  summarize  our  accomplishments  under  Grant  FA9550-12-1-0287.  This 
project  has  two  interrelated  research  areas.  The  first  of  these  focuses  on  scalable,  high- 
performance  inference  algorithms  for  graphical  and  hierarchical  models.  This  has  clear 
applications  to  sensor  exploitation  applications  such  as  tracking  and  distributed  network  fusion, 
including  the  development  of  message-passing  algorithms  for  location-aware  networks  in 
complex  (possibly  GPS-denied)  and  often  communications-limited  environments.  Our  second 
thrust  focuses  on  discovering  graphical  models  not  only  relating  different  sensor  observables  but 
also  discovering  and  linking  them  to  higher-level  “hidden”  variables  capturing  the  common 
context  that  relates  them.  One  motivation  here  is  to  enhance  both  lower-level  sensor  processing 
(e.g.,  for  object  recognition)  and  higher-level  context  discovery  through  these  models.  A  second 
motivation  is  the  discovery  of  complex,  possibly  coordinated  dynamic  behavior  exploiting 
emerging  methods  of  Bayesian  nonparametric  modeling.  Our  research  blends  methods  from 
statistics  and  probabilistic  modeling,  signal  and  image  processing,  communications  and 
information  theory,  optimization,  mathematical  physics,  graphical  models,  and  machine  learning 
theory,  yielding  new  approaches  to  challenging  problems  in  sensing,  surveillance,  and 
navigation.  These  are  areas  of  clear  and  direct  relevance  to  Air  Force  and  more  broadly  DoD 
missions,  and  some  of  our  methods  are  already  impacting  AF-  and  DoD-sponsored  programs 
with  industry. 

The  principal  investigator  for  this  effort  is  Professor  Alan  S.  Willsky  and  the  Co-Principal 
Investigator  is  Professor  Moe  Z.  Win.  Professors  Willsky  and  Win  are  assisted  in  the  conduct  of 
this  research  by  several  graduate  research  assistants  as  well  as  additional  thesis  students  not 
requiring  stipend  or  tuition  support  from  this  grant.  In  the  next  section  we  briefly  describe  our 
research  efforts;  in  Section  III  we  indicate  the  individuals  involved  in  this  effort;  in  Section  IV 
we  list  the  publications  supported  by  this  effort;  and  in  Section  V  we  discuss  several  other  topics 
including  honors  received  by  researchers  involved  in  this  project  and  transitions  of  our  research. 
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II.  Accomplishments 


In  this  section  we  briefly  describe  our  research  under  this  grant.  We  limit  ourselves  here  to  a 
succinct  summary  and  refer  to  the  publications  listed  at  the  end  of  this  report  for  detailed 
developments. 

2.1  Efficient  and  Scalable  Sensor  Fusion  and  Exploitation  Algorithms  Based  on  Graphical 
and  Hierarchical  Models 

The  overall  objective  of  this  portion  of  our  research  is  the  development  of  methods  for 
identifying  classes  of  stochastic  models  for  phenomena  that  vary  over  space,  time,  and  hierarchy 
and  that  possess  structure  which  can  be  exploited  to  construct  efficient  and  scalable  algorithms 
for  statistical  inference. 

One  important  problem  in  machine  learning  approaches  to  “big  data”  problems  in  signal 
processing  is  the  development  of  computationally  efficient  and  scalable  algorithms  for  Markov 
Chain  Markov  Chain  (MCMC)  sampling  and  in  particular  the  sampling  of  very  high-dimensional 
Gaussian  processes.  In  many  cases  these  processes  are  specified  in  terms  of  so-called  graphical 
models  that  capture  the  dependency  structure  among  the  variables  in  a  Gaussian  process. 
However,  even  if  the  resulting  graphs  are  sparse  sampling  of  these  processes  has  been  a 
challenge,  except  for  graphs  with  particular  structure  (such  as  acyclic  graphs  or  graphs  with 
small  feedback  vertex  sets  (see  discussion  to  follow)).  We  have  succeeded  in  developing  two 
very  powerful  approaches  to  overcoming  this  challenge  for  models  on  rich  set  of  graphs.  The 
first  of  these  applies  for  graphs  that  have  the  property  that  removing  a  relatively  small  set  of 
edges  exposes  a  graph  with  tractable  sampling  structure.  In  this  case,  one  can  use  a  so-called 
low-rank  subgraph  perturbation  to  perform  iterative  sampling  that  provably  converges  to 
provide  samples  from  the  full  graphical  model.  For  any  graph  that  has  this  structure  -  and  this 
includes  quite  a  rich  set  of  graphs  -  there  typically  are  many  different  choices  of  small  sets  of 
edges  that  can  be  removed  to  expose  the  desired  structure,  and  by  switching  among  these 
different  sets  (which  means  perturbing  different  subgraphs  in  each  iteration)  one  can  generally 
achieve  even  faster  convergence.  The  second  of  our  approaches  provides  only  approximate 
samples  of  the  model  but  does  so  in  a  manner  that  allows  highly  parallelizable  algorithms  -  in 
essence  by  partitioning  the  graph  into  several  components,  performing  parallel  sampling  within 
each  component  and,  at  a  slower  rate,  updating  the  variables  involved  in  edges  that  connect  those 
components.  This  is,  in  essence,  a  mixture  of  Jacobi-like  iterations  with  Gauss-Seidel  iterations, 
and  as  such,  under  very  mild  conditions  it  produces  the  correct  mean  values  for  all  variables. 
The  approximations  that  occur  involve  the  covariances  among  the  variables  (due  to  the  lower 
rate  of  updating  across  the  components),  and  we  have  characterized  in  detail  that  approximation, 
allowing  one  to  see  the  tradeoff  between  speed  and  accuracy  (where  speed  is  sacrificed  at  the 
expense  of  accuracy  if  such  cross-component  updates  occur  more  frequently,  and  the  opposite 
occurs  if  updates  occur  less  frequently). 

A  second  important  part  of  our  research  has  involved  a  thorough  exploitation  of  the  structure 
of  a  particular  class  of  Gaussian  graphical  models,  namely  those  that  have  comparatively  small 
Feedback  Vertex  Sets  (FVS’s).  An  FVS  is  a  set  of  nodes  that,  if  eliminated  leave  a  cycle  free 
graph.  For  any  graphical  model  with  an  identified  FVS,  exact  computation  of  means  and 
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variances  can  be  accomplished  by  first  performing  cyele-free  eomputation  on  the  eyele-free 
graph  exposed  by  eliminating  the  FVS  nodes,  then  computing  means  and  eovarianees  for  the 
FVS  nodes,  followed  by  a  seeond  round  of  cyele-free  eomputations  to  yield  means  and  variances 
at  the  remaining  nodes.  Roughly  speaking  this  is  equivalent  to  Gaussian  elimination  of  the 
variables  in  the  eyele-free  graph,  followed  by  computations  on  the  FVS,  and  then  back- 
substitution  to  the  variables  in  the  cyele-free  graph.  For  the  first  and  third  step  of  this  proeess  the 
eomputations  ean  be  performed  in  a  completely  parallelized  manner,  through  message-passing, 
using  so-ealled  Belief  Propagation,  which  converges  in  a  finite  number  of  steps  eorresponding  to 
the  diameter  of  the  eyele-free  graph,  so  that  the  potential  computational  bottleneck  is  solely  in 
the  eomputation  on  the  FVS  itself,  where  brute-force  solution  yields  computations  that  are  eubic 
in  the  eardinality  of  the  FVS.  Henee,  this  approach  works  well  in  yielding  exaet  solutions  if  the 
size  of  the  FVS  is  relatively  small.  If  this  is  not  the  case,  one  ean  use  an  approximate  solution,  in 
whieh  we  form  a  pseudo-FVS  -  i.e.,  a  set  of  nodes  that  when  removed  yield  a  graph  that  still  has 
some  cyeles  -  together  with  performing  so-ealled  Loopy  Belief  Propagation  (LBP)  in  the 
remaining  graph.  In  this  case  LBP  isn’t  always  guaranteed  to  eonverge  (and  when  it  does 
eonverge  does  so  asymptotically),  but  we  have  developed  a  very  simple  approach  to  choosing  the 
nodes  for  the  pseudo-FVS  that  guarantees  eonvergence,  and  we  have  also  characterized  precisely 
what  the  approximation  is  that  such  an  algorithm  makes.  In  partieular,  the  eomputation  of  the 
mean  is  eorreet  for  all  variables,  the  varianees  are  also  eorreet  for  the  pseudo-FVS  nodes,  and  the 
approximation  of  the  varianees  for  the  other  nodes  ean  be  preeisely  characterized  in  terms  of  the 
so-ealled  “weighted  walks”  that  this  algorithm  misses.  We  have  demonstrated  the  effectiveness 
of  this  algorithm  (and  its  superior  performanee  to  other  available  teehniques)  on  a  number  of 
different  graphs. 

One  of  the  advantages  of  LBP  applied  to  any  graph  is  that  the  algorithm  structure  involves 
only  local  graph  properties  -  i.e.,  the  eomputations  at  each  node  involves  only  its  immediate 
neighbors  together  with  the  weights  on  the  edges  that  connect  the  node  to  its  neighbors,  so  that 
the  algorithm  is  completely  distributed,  where  the  only  “protoeol”  information  that  is  included  in 
the  “messages”  passed  between  nodes  is  the  identity  of  the  node  sending  the  message.  The 
algorithm  just  described,  however,  appears  to  require  mueh  more  global  eomputation  (both  to 
determine  what  nodes  to  inelude  in  the  (pseudo-)  FVS  and  then  to  perform  the  computations 
among  the  variables  in  that  set.  We  have  now  developed  a  eompletely  distributed  of  our 
algorithm  that  simultaneously  (a)  ehooses  the  nodes  to  inelude  in  the  (pseudo-)  FVS;  (b) 
performs  message-passing  among  nodes  that  have  been  eonfirmed  as  not  being  in  the  (pseudo-) 
FVS;  and  (c)  deeides  on  the  fly,  node-by-node,  when  a  node  in  the  (pseudo-)  FVS  should  be 
“released”  to  allow  it  to  join  the  message  passing.  This  algorithm  is  completely  distributed, 
albeit  with  some  additional  overhead,  as  eaeh  node  must  provide  some  additional  protoeol 
information  (first  to  establish  if  it  should  “join”  the  pseudo-FVS  and  then  its  status  as  “active,” 
i.e.,  involved  in  the  message  passing,  or  not  yet  aetive). 

One  important  part  of  our  work  is  the  study  for  effieient  fusion  of  sensory  data  and 
environmental  information  for  network  loealization  and  navigation  in  harsh  propagation 
environments.  Conventional  filtering  teehniques  fail  to  provide  satisfaetory  performance  in 
many  important  nonlinear  or  non-Gaussian  scenarios.  In  addition,  there  is  a  lack  of  a  unified 
methodology  for  the  design  and  analysis  of  different  filtering  teehniques.  To  address  these 
problems,  we  have  proposed  a  new  filtering  methodology  called  belief  eondensation  (BC) 
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filtering.  Moreover,  we  have  developed  BC-based  teehniques  for  representing  the  positional 
belief  in  cooperative  navigation  networks.  Such  a  method  can  reduce  the  communication 
overhead  of  network  cooperation.  The  simulation  results  validated  that  the  proposed  techniques 
can  handle  the  nonlinear  dynamics  and  non-Gaussian  beliefs  with  an  improved  accuracy  versus 
complexity  tradeoffs  than  conventional  techniques. 

The  BC  filtering  techniques  are  applied  in  the  design  of  localization  and  navigation 
algorithms.  We  have  developed  a  framework  for  context-aided  inertial  navigation  and  efficient 
algorithms  for  its  implementation  based  on  belief  condensation.  The  results  presented  provide  a 
principled  methodology  to  merge  information  from  inertial  measurements  and  situational  context 
through  Bayesian  inference  over  an  augmented  hidden  Markovian  model  (HMM).  In  addition, 
the  techniques  developed  enable  efficient  and  accurate  context  modeling  and  data  fusion  through 
belief  condensation.  The  results  obtained  through  experimentation  for  the  case  study  of 
pedestrian  foot-mounted  IMU  show  that  the  proposed  techniques  can  outperform  existing 
techniques  while  maintaining  complexities  that  meet  the  real-time  requirements.  The  theoretical 
framework  and  algorithmic  techniques  presented  can  enable  the  integration  of  contextual 
information  in  inertial  navigation  systems  and  hence  improve  localization  accuracy  and 
robustness. 

Another  important  part  of  our  work  is  the  study  for  joint  design  of  location  inference  and 
power  control  strategies.  These  two  operations  of  network  navigation  interrelate  with  each  other, 
thus  motivating  the  design  of  joint  inference  and  control  algorithms.  We  first  determine  the 
confidence  region  for  location  inference  based  on  Fisher  information  analysis,  and  develop 
robust  power  control  strategies  to  minimize  the  position  errors  of  the  agents  within  the 
confidence  region.  In  the  centralized  setting,  we  show  that  the  power  control  strategy  can  be 
transformed  semi-definite  programs.  In  the  distributed  setting,  we  decompose  the  problem  into  a 
sequential  two-phase  problem  to  circumvent  the  difficulties  due  to  the  lack  of  global  knowledge. 
We  show  that  the  power  control  strategy  in  both  phases  can  be  transformed  into  second-order 
cone  programs.  The  computation  complexity  of  the  proposed  algorithm  is  analyzed  using  the 
sparsity  property  of  the  resource  allocation.  Simulation  results  show  that  the  proposed  strategies 
significantly  improve  localization  accuracy  and  robustness  compared  to  the  conventional 
strategies. 

2,2  Discovery  of  Graphical  and  Hidden  Hierarchical,  Contextual,  and  Dynamic 
Relationships  in  Complex  Sensor  Data 

The  research  described  in  this  section  deals  with  methods  for  discovering  latent  structure  in 
complex  sensor  data,  where  that  latent  structure  can  take  the  form  of  hierarchical  graphical 
models  of  different  types.  Our  research  has  led  to  the  following  lines  of  inquiry  and  results: 

As  described  in  the  preceding  section,  there  are  very  powerful  estimation/signal  processing 
algorithms  that  can  be  employed  for  graphical  models  with  Feedback  Vertex  Sets  of  moderate 
size.  This  leads  naturally  to  the  question  of  whether  one  can  learn  models  with  small  FVS’s 
from  data  or  can  approximate  specified  covariance  structure  of  a  large  Gaussian  process  with  a 
model  that  has  a  comparatively  small  FVS.  For  cycle-free  graphs  there  is  a  very  powerful 
algorithm,  known  as  the  Chow-Liu  (CL)  algorithm  to  determine  either  the  maximum-likelihood 
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estimate  of  a  eyele-free  model  (including  the  structure  of  the  cycle-free  graph)  or,  equivalently, 
the  cycle-free  model  that  is  closest  in  Kullback-Leibler  (K-L)  divergence  to  a  specified 
covariance.  We  have  developed  a  suite  of  algorithms  that  exploit  the  CL-algorithm  to  yield  the 
structure  of  Gaussian  graphical  models  with  (small)  FVS’s.  The  simplest  of  these  algorithms  is 
one  in  which  we  pre-specify  the  set  of  nodes  to  include  in  an  FVS.  The  next  algorithm  is  one 
that  can,  in  principle,  determine  the  FVS  nodes  and  the  structure  of  the  remaining  (cycle-free) 
graph.  This  algorithm  has  potential  combinatorial  issues  as  one,  in  principle,  needs  to  search 
over  all  subsets  of  nodes  up  to  a  specified  cardinality.  Instead  we  have  implemented  and  tested  a 
greedy  algorithm  that  adds  FVS  nodes  one  at  a  time,  choosing  nodes  that  provide  the  best 
approximation  one  by  one.  In  addition,  we  have  developed  an  algorithm  in  which  the  FVS  nodes 
are  all  hidden  or  latent,  i.e.,  they  are  nodes  (and  associated  variables)  that  are  added  to  the 
overall  process  so  that  (a)  conditioned  on  these  nodes,  the  set  of  variables  we  wish  to  model  have 
a  graphical  structure  that  is  acyclic;  while  (b)  the  marginal  distribution  on  that  set  of  variables, 
i.e.,  when  we  marginalize  out  the  latent  variables  in  order  to  reduce  the  K-L  divergence  or  the 
resulting  model  from  the  specified  covariance.  This  is  an  iterative  algorithm,  which  exploits  the 
algorithms  we  have  developed  for  the  case  of  a  known  FVS,  and  which  we  have  proved  does, 
indeed  decrease  K-L  divergence  at  each  iteration.  Left  open  at  this  point  is  analyzing  if  and 
when  the  algorithm  actually  converges  to  the  optimum  in  terms  of  minimizing  K-L  divergence. 

We  have  also  developed  new  algorithms  and  rich  sets  of  models  involving  Bayesian 
nonparametric  methods.  Such  models,  exploiting  Dirichlet  processes,  allow  model  complexity  to 
be  inferred  at  the  same  time  as  learning  the  parameters  of  such  models.  This  has  been  the  subject 
of  considerable  effort  in  the  machine  learning  community,  and  our  major  contributions  have  been 
in  developing  methods  for  learning  models  for  dynamic  phenomena.  Our  first  step  in  this 
development  was  the  develop  methods  for  learning  Hidden  Markov  Models  {HMM’s)  for  signals, 
where  included  in  the  learning  process  is  inferring  the  number  of  states  in  the  HMM  as  well  as 
the  transition  behavior  among  these  states  and  the  distribution  of  signal  values  when  the  HMM  is 
in  each  of  those  states.  This  has  been  extended  to  a  model  with  an  additional  level  of  hierarchy 
in  which  each  HMM  state  specifies  a  dynamic  model  (e.g.,  an  autoregressive  time  series  or  state 
space  model)  whose  output  generates  observed  signal  values.  One  of  the  significant  limitation  of 
these  HMM -based  models  is  that  HMM’s  must  have  geometric  holding  times  in  each  state, 
which  is  not  the  case  for  many  phenomena  of  interest.  Hence  we  have  explored  and  developed 
approaches  that  replace  HMM’s  with  Hidden  ^emz-Markov  Processes  (HSMM’s)  -  models  in 
which  the  sequence  of  hidden  discrete  states  is,  indeed,  Markov,  but  in  which  the  holding  time  in 
each  state  can  be  distributed  in  a  far  richer  -  and  much  more  realistic  -  way.  One  of  the 
challenges  with  such  models  (with  either  HMM’s  or  HSMM’s)  is  computational.  We  have 
developed  methods  along  two  lines.  The  first  of  these  involves  Gibbs  sampling.  Here  we  have 
exploited  two  different  ideas.  One  involves  introducing  a  rich  but  easily  parameterized  class  of 
holding  time  distributions,  namely  so-called  negative  binomial  distributions,  so  that  we  can 
streamline  considerably  the  sampling  of  this  part  of  the  model.  The  other  is  to  do  approximate 
parallel  sampling  with  occasional  coordination  among  the  steps,  rather  than  the  strictly  required 
ordering  in  which  sampling  of  different  elements  of  a  model  must  be  done  in  order  to  do  true 
Gibbs  sampling  (indeed,  this  is  the  generalization  of  the  approach  to  approximate  parallel 
sampling  of  large  Gaussian  models  described  in  the  preceding  section;  in  fact,  it  was  this 
challenge  with  nonparametric  model  sampling  that  motivated  the  analysis  of  the  Gaussian  case). 
The  second  method  that  we  developed  for  inference  for  these  complex  models  is  the 
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development  of  so-ealled  variational  methods  for  updating  of  parameters  of  these  models  -  these 
involve  approximations  analogous  to  mean-field  but  for  the  eomplex  nonparametrie  models 
considered  in  this  portion  of  our  work.  These  methods  have  received  a  great  deal  of  attention 
both  from  other  academic  researchers  including  those  involved  in  animal  behavior  analysis  and 
from  those  involved  in  applied  programs  for  DoD  and  other  governmental  agencies  (see  Section 
V). 


We  have  also  performed  extensive  research  on  optimization-based  methods  for  a  variety  of 
problems.  These  have  involved  convex  and  semi-definite  relaxations  of  difficult  optimization 
problems.  Our  work  has  included  theoretical  advances  in  terms  of  new  algorithms  as  well  as 
theoretical  guarantees.  We  have  also  developed  new  algorithms  for  problems  of  estimation  on 
complex  objects,  including  and  in  fact  motivated  by  problems  of  attitude  estimation.  In  addition, 
we  have  returned  to  a  problem  that  was  originally  motivated  by  multiresolution  signal  analysis  - 
inspired  by  the  emergence  of  wavelets  as  an  important  signal  processing  tool.  In  earlier  work  we 
had  developed  estimation  algorithms  for  stochastic  models  that  lived  on  multiresolution  trees 
(which  in  turn  provided  the  initial  motivation  for  our  work  on  graphical  models  more  generally). 
We  had  also  developed  methods  for  learning  models  for  such  signals  in  which  the  variables  at 
coarser  resolutions  than  the  actual  signal  or  phenomenon  were  viewed  as  hidden  variables.  In 
particular,  in  our  earliest  work  in  this  area  we  developed  methods  analogous  to  those  used  to 
estimate  state  space  models  for  constructing  variables  at  each  of  the  hidden  nodes  (corresponding 
to  coarser  representations  of  the  signal  of  interest)  where  the  structure  of  the  multi-resolution  tree 
was  fixed  but  the  dimensions  of  the  variables  at  the  hidden  nodes  was  unknown  and  needed  to  be 
learned  (in  our  case  using  ideas  from  canonical  correlations  and  predictive  efficiency).  In 
subsequent  work  we  developed  methods  for  the  complement  of  this  earlier  work,  in  which  we 
restricted  each  of  the  hidden  variables  to  be  of  the  same  dimension  (namely  scalar)  as  the 
individual  values  of  the  observed  signal,  but  where  the  structure  of  the  hierarchical  tree  was  to  be 
learned.  Most  recently,  we  have  taken  on  what  appears  to  be,  and  indeed  is,  a  very  ill-posed 
problem,  namely  learning  both  the  structure  of  the  multi-resolution  tree  as  well  as  the  dimensions 
of  the  variables  at  every  hidden  node.  By  exploiting  an  optimization-based  framework  with 
ideas  analogous  to  those  used  in  sparse  signal  recovery  we  have  developed  a  method  to  solve  this 
problem.  In  the  process,  we  have  identified  conditions  under  which  exact  recovery  is  guaranteed 
(if  an  exact  multi-resolution  model  exists  and  we  have  precise  covariance  information).  This 
condition  essentially  corresponds  to  excluding  what  we  had  previously  called  internal  models  in 
which  the  hidden  variables  were  actually  deterministic  functionals  of  the  observed  signal.  Given 
noisy  data,  that  situation  is  a  singular  one,  so  that  this  restriction  is  of  no  practical  consequence. 

We  have  developed  statistical  techniques  for  localization  in  harsh  propagation  environments. 
Non-line-of-sight  (NLOS)  propagation,  multipath  effect,  and  multiuser  interference  significantly 
affect  the  localization  accuracy.  Many  techniques  have  been  proposed  to  address  this  problem; 
most  of  them  focus  on  improving  the  accuracy  of  ranging  estimation,  e.g.,  NLOS  identification 
and  mitigation.  We  have  taken  one  step  further  and  introduced  the  concept  of  range  likelihood 
(RL)  of  a  set  of  range-related  measurements.  We  show  that  such  RLs,  instead  of  range  point 
estimate,  encapsulate  all  the  measurement  information  that  is  needed  to  perform  Bayesian.  Then, 
we  present  effective  techniques  to  estimate  RLs  based  on  generative  model  estimation,  i.e, 
density  estimation.  We  have  assessed  the  performance  of  our  proposed  approaches  by  using 
experimental  data  from  an  indoor  measurement  campaign  with  FCC-compliant  ultra-wideband 
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radios.  The  results  show  that  the  proposed  approaehes  can  significantly  improve  the 
performance  of  wireless  localization  in  harsh  propagation  environments. 

We  have  derived  the  performance  bounds  for  orthogonal  frequency-division  multiplexing 
(OFDM)  ranging  based  on  Fisher  information  analysis.  In  particular,  we  have  decomposed  the 
Fisher  information  matrix  (FIM)  for  parameter  estimation  of  multipath  channels  into  the  sum  of 
those  FIMs  for  different  subcarriers.  Based  on  the  equivalent  Fisher  information  analysis,  we 
analyzed  the  effects  of  signal  and  channel  parameters  on  the  OFDM  ranging  accuracy.  Our 
results  provide  the  quantitative  relationship  between  the  ranging  accuracy  and  the  central 
frequency,  the  subcarrier  spacing,  the  number  of  subcarriers,  and  the  power  allocation  across  the 
subcarriers.  These  results  can  serve  as  guidelines  for  designing  OFDM  ranging  systems. 

We  have  studied  the  spatiotemporal  information  coupling  effects  in  cooperative  network 
navigation.  In  cooperative  navigation  networks,  the  correlation  of  the  inferred  nodes’  positions 
leads  to  the  concept  of  information  coupling.  It  hinders  the  development  of  distributed 
techniques  that  account  for  such  correlation  since  the  monitoring  of  neighbors’  interaction 
requires  difficult  network  coordination.  We  have  developed  a  framework  to  characterize  such 
effects  using  Fisher  information  analysis.  In  particular,  we  have  derived  the  information 
coupling  term  in  the  equivalent  Fisher  information  matrix  (EFIM)  for  individual  positional  states 
and  characterized  the  effective  information  obtained  from  each  neighbor.  Our  results  in  this  area 
show  that  the  effect  of  information  coupling  sharply  decreases  with  the  network  hopping 
distance.  This  local  nature  of  information  coupling  can  be  exploited  by  cooperative  techniques 
since  the  information  coupling  induced  by  distant  neighbors’  interactions  can  be  safely  ignored. 
The  results  will  serve  as  guidelines  for  the  design  of  efficient  and  accurate  techniques  for 
network  navigation. 

We  have  also  studied  resource  allocation  in  network  localization  based  on  the  computational 
geometry  framework.  To  achieve  the  best  localization  performance,  we  have  developed  a 
computational  geometry  framework  for  optimal  resource  allocation  in  WNL.  We  first  determine 
an  affine  map  that  transforms  each  resource  allocation  strategy  into  a  point  in  3-D  Euclidian 
space.  By  exploiting  geometric  properties  of  these  image  points,  we  prove  the  sparsity  property 
of  the  optimal  resource  allocation  vector,  i.e.,  the  optimal  localization  performance  can  be 
achieved  by  allocating  resources  to  only  a  small  subset  of  neighboring  nodes.  Moreover,  these 
geometric  properties  enable  the  reduction  of  the  search  space  for  optimal  solutions.  This  allows 
us  to  design  efficient  resource  allocation  strategies.  Numerical  results  show  that  the  proposed 
strategies  can  achieve  significant  improvements  in  both  localization  performance  and 
computation  efficiency. 


9 

DISTRIBUTION  A:  Distribution  approved  for  public  release. 


III.  Personnel 


The  following  is  a  list  of  individuals  who  have  worked  on  researeh  supported  in  whole  or  in 
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Mr.  Matthew  Johnson,  graduate  student  (Ph.D.  completed) 
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Mr.  Tianheng  Wang,  graduate  student  (Ph.D.  in  progress) 

Mr.  Wenhan  Dai,  graduate  student  (Ph.D.  in  progress) 
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IV.  Publications 


The  publications  listed  below  represent  papers,  reports,  and  theses  supported  in  whole  or  in 

part  by  the  Air  Force  Office  of  Scientific  Research  under  Grant  FA9550-12-1-0287: 

[1]  V.  Chandrasekaran,  P.A.  Parrilo,  and  A.S.  Willsky,  “Latent  Variable  Graphical  Model 
Selection  via  Convex  Optimization,”  Annals  of  Statistics,  Vol.  40,  No.  4,  Aug.  2012,  pp. 
1935-1967  (with  special  designation  as  a  “discussion  paper). 

[2]  Y.  Liu,  V.  Chandrasekaran,  A.  Anandkumar,  and  A.S.  Willsky,  “Feedback  Message 
Passing  for  Inference  in  Gaussian  Graphical  Models,”  IEEE  Trans,  on  Signal  Processing, 
Vol.  60,  No.  8,  Aug.  2012. 

[3]  A.  Anandkumar,  V.Y.F.  Tan,  and  A.S.  Willsky,  “High-Dimensional  Robust  Structure 
Learning  of  Ising  Models  on  Sparse  Random  Graphs,”  Annals  of  Statistics.  Vol.  40,  No.  3, 
Jun.  2012,  pp. 1346-1375. 

[4]  V.  Chandrasekaran,  B.  Recht,  P.  Parrilo,  and  A.S.  Willsky.  “The  Convex  Geometry  of 
Linear  Inverse  Problems,”  Foundations  of  Computational  Mathematics,  Vol.  12,  No.  6, 
Dec.  2012. 

[5]  V.  Chandrasekaran,  P.  Parrilo,  and  A.S.  Willsky.  “Convex  Graph  Invariants,”  SIAM 
Review,  Vol.  54,  No.  3,  Sept.  2012. 

[6]  A.  Anandkumar,  V.Y.F.  Tan,  and  A.S.  Willsky,  “High-Dimensional  Gaussian  Graphical 
Model  Selection:  Walk-Summability  and  Local  Separation  Criterion,”  J.  of  Machine 
Learning  Research,  Vol.  13,  Aug.  2012,  pp.  2293-2331 . 

[7]  Y.  Liu,  O.  Kosut,  and  A.S.  Willsky,  “Sampling  GMRFs  by  Subgraph  Correction,”  NIPS 
Workshop  on  Perturbation,  Optimization,  and  Statistics,  2012. 

[8]  J.Saunderson,  P.A.  Parrilo,  and  A.S.  Willsky,  “Diagonal  and  Low-Rank  Decompositions 
and  Fitting  Ellipsoids  to  Random  Points,”  IEEE  Conf.  on  Decision  and  Control,  Dec. 
2013. 

[9]  Y.  Liu,  O.  Kosut,  and  A.S.  Willsky,  “Sampling  from  Gaussian  Graphical  Models  Using 
Subgraph  Perturbation,”  IEEE  Int’l  Symp.  on  Information  Theory,  July  2013. 
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Models,”  IEEE  Int’l  Symp.  on  Information  Theory,  July  2013. 

[11]  S.  Mazuelas,  Y.  Shen,  and  M.  Z.  Win,  “Belief  Condensation  Filtering,”  IEEE  Trans,  on 
Signal  Processing,  vol.  61,  no.  18,  Sept.  2013,  pp.  4403-4415. 

[12]  F.  Montorsi,  S.  Mazuelas,  G.  M.  Vitetta,  and  M.  Z.  Win,  “On  the  Performance  Limits  of 
Map-Aware  Localization,”  IEEE  Trans,  on  Information  Theory,  vol.  59,  no.  8,  Aug.  2013, 
pp.  5023-5038. 

[14]  T.  Wang,  Y.  Shen,  S.  Mazuelas,  H.  Shin,  and  M.  Z.  Win,  “On  OFDM  ranging  accuracy  in 
multipath  channels,”  IEEE  Systems  Journal,  vol.  8,  no.  1,  Mar.  2014,  pp.  104-114. 

[15]  Johnson,  M.J.  and  Willsky,  A.S.,  “Bayesian  Nonparametric  Hidden  Semi-Markov 
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[16]  MJ.  Johnson,  J.  Saunderson,  and  A.S.  Willsky,  “Analyzing  Hogwild  Parallel  Gaussian 
Gibbs  Sampling,”  Neural  Information  Proeessing  Systems,  Dec.  2013. 

[17]  Y.  Liu  and  A.S.  Willsky,  “Gaussian  Graphical  Models  with  Small  FVSs:  Trading  off 
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V.  RECOGNITION/INTERACTIONS/TRANSITIONS 


In  this  section  we  summarize  our  and  plans  for  transitions  associated  with  research 
supported  by  AFOSR  Grant  FA9550-12-1-0287,  as  well  as  listing  some  important  honors 
received  by  members  of  our  research  team. 

Honors  and  Recognition 

1)  Prof  Willsky’s  student,  Myungjin  Choi,  received  a  Sprowls  Best  Computer  Science 
Thesis  award  from  MIT’s  Department  of  Electrical  Engineering  and  Computer  Science. 

2)  Prof.  Willsky’s  student,  Venkat  Chandrasekaran,  received  a  Jin  Au  Kong  Best  Electrical 
Engineering  Thesis  award  from  MIT’s  Department  of  Electrical  Engineering  and 
Computer  Science. 

3)  Prof.  Willsky  was  selected  as  the  2013  William  Gould  Dow  Distinguished  Eecturer  at  the 
Dept,  of  EECS,  University  of  Michigan.  Prof.  Willsky  gave  his  Dow  lecture  in  January 
2013. 

4)  Prof.  Willsky  was  selected  as  the  2013  Dean  Eytle  Electrical  Engineering  Endowed 
Eecturer  at  the  University  of  Washington.  Prof.  Willsky  gave  his  Eytle  lectures  in  May 
2013. 

5)  Prof.  Win  together  with  his  students  Yuan  Shen,  Wesley  M.  Gifford,  and  postdoctoral 
fellow  Santiago  Mazuelas  received  the  IEEE  Communications  Society  Ered  W.  Ellersick 
Prize,  2012. 

6)  Prof.  Win  and  his  student  Wesley  M.  Gifford  received  the  IEEE  Communications  Society 
Stephen  O.  Rice  Prize  in  the  Eield  of  Communications  Theory,  2012. 

7)  Prof.  Win  together  with  his  postdoctoral  fellow  Alberto  Rabbachin  received  IEEE 
Communications  Society  William  R.  Bennett  Prize  in  the  Eield  of  Communications 
Networking,  2012. 

8)  Prof  Win  received  Institute  of  Advanced  Study  Natural  Sciences  and  Technology 
Eellowship  to  give  an  invited  lecture  at  the  Institute  of  Advanced  Study,  University  of 
Bologna,  Bologna,  Italy,  in  July  2012. 

9)  Prof.  Win  together  with  his  students  Henghui  Eu  and  postdoctoral  associate  Santiago 
Mazuelas  received  a  Best  Paper  Award  at  the  IEEE  International  Conference  on 
Communications  in  June  2013. 

10)  Prof.  Win  was  named  as  recipient  of  the  2013  International  Prize  for  Communications 
Cristoforo  Colombo,  established  by  the  city  of  Genova  in  1954. 

11)  Prof.  Willsky  was  named  as  recipient  of  the  2014  SPS  Society  Award,  the  highest  award 
bestowed  by  the  IEEE  Signal  Processing  Society. 

12)  Prof.  Win  was  named  as  recipient  of  the  2014  Technical  Recognition  Award  of  the  IEEE 
ComSoc  Signal  Processing  and  Communications  Electronics  (SPCE)  Technical 
Committee. 

13)  Prof.  Win  together  with  his  student  Wenhan  Dai  and  postdoctoral  fellow  Stefania 
Bartoletti  received  a  Student  Paper  Award  (first  place)  at  the  IEEE  Canadian  Workshop 
on  Information  Theory  in  July  2015. 

14)  A  2-day  symposium  in  Prof.  Willsky’s  honor  was  held  at  MIT  on  March  18-19,  2016. 
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Participation/Presentation  at  Meetings 


In  addition  to  the  invited  and  eontributed  papers  presented  at  various  meetings,  we  also  make 
note  of  the  following: 

1)  Prof.  Willsky  gave  a  plenary  leeture  at  the  IF  AC  System  Identifieation  Conferenee  in 
Brussels,  Belgium  in  July  2012. 

2)  Prof.  Win  gave  a  presentation  on  “Network  loealization  and  navigation  (a  strategie 
advantage  in  modem  warfare),”  to  U.S.  Navy  Naval  Undersea  Warfare  Center  (NUWC), 
Naval  Sea  Systems  Command  (NAVSEA)  through  the  MIT  Industrial  Liaison  Program 
Seminar,  Cambridge,  MA,  Oetober  2012. 

3)  Prof.  Willsky  was  the  2013  William  Gould  Dow  Distinguished  Leeturer,  the  most 
prestigious  invited  leeture  in  the  Dept,  of  EECS,  University  of  Miehigan. 

4)  Prof.  Willsky  was  the  2013  Dean  Lytle  Eleetrieal  Engineering  Endowed  Leeturer,  the 
most  prestigious  invited  leeture  in  the  Dept,  of  EE,  University  of  Washington. 

5)  Prof.  Win  gave  a  keynote  address  at  the  IEEE  International  Conferenee  on  Wireless 
Communieations  and  Signal  Proeessing,  in  Hangzhou,  China,  in  Oetober  2013. 

6)  Prof.  Win  gave  an  Invited  University  Lecture  at  the  100th  Anniversary  Event  of  the 
Chulalongkorn  University  in  Bangkok,  Thailand,  in  June  2013. 

7)  Prof.  Win  gave  a  keynote  address  at  the  IEEE  International  Conference  on 
Communications,  Workshop  on  Advances  in  Network  Localization  and  Navigation 
(ANLN)  in  Budapest,  Hungary,  in  June  2013. 

8)  Prof.  Win  gave  a  keynote  address  at  the  International  Symposium  on  Medical 
Information  and  Communication  Technology  (ISMICT),  Elorence,  Italy,  in  April  2014. 

9)  Prof.  Win  gave  a  tutorial  on  “Network  localization  and  navigation:  from  theory  to 
practice”  at  the  IEEE  Wireless  Communications  and  Networking  Conference  in  New 
Orleans,  LA,  in  March  2015. 

10)  Prof.  Win  gave  a  keynote  address  at  the  International  Conference  on  Localization  and 
GNSS  in  Gothenburg,  Sweden,  in  June  2015. 

11)  Prof.  Win  gave  a  keynote  address  at  the  IEEE  Canadian  Workshop  on  Information 
Theory  in  St.  John’s,  NL,  Canada,  in  July  2015. 

12)  Prof.  Win  gave  a  plenary  presentation  at  the  IEEE  International  Conference  on 
Communications  in  China  in  Shenzhen,  in  November  2015. 


Consultative  and  Advisory  Functions 

We  continue  to  be  actively  engaged  in  a  number  of  activities  relevant  to  the  research  being 
performed  under  our  AEOSR  grant: 

(1)  Prof.  Willsky  has  acted  as  a  consultant  to  Parietal  Systems  (PSI),  and  continues  to  consult 
with  Systems  and  Technology  Research  (STR)  on  a  number  of  research  projects 
including  ones  that  represent  direct  transitions  of  the  technology  being  developed  under 
our  AEOSR  Grant. 

(2)  In  his  role  as  a  consultant  to  STR,  Prof.  Willsky  served  as  one  of  the  organizers  and 
leader  of  the  panel  discussion  at  the  AERL-sponsored  Workshop  on  Multi-INT  Eusion  for 
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Contested  Environments,  held  in  Dayton,  OH  in  August  2014  (STR  Contact:  Dr.  Mark 
Luettgen  (mark.luettgen@stresearch.com);  AFRL  Contacts:  Mr.  Phillip  Hanselman 
(phillip.hanselman@us.af.mil),  Mr.  Edmund  Zelnio  (edmund.zelnio@us.af.mil),  and  Dr. 
Juan  Vasquez  (juan.vasquez@us.afmil). 

Transitions 

The  following  represent  some  of  the  transitions  of  our  work. 

1)  Our  methods  for  nonparametric  learning  and  graphical  models  are  being  adapted  and 

applied  at  STR  to  problems  in  multi-object  tracking  and  multi-INT  fusion  under  a 
program  sponsored  by  AFRL  Dayton.  The  STR  contacts  are  Dr.  Mark  Luettgen 
(mark.luettgen@stresearch.com)  and  Dr.  Stefano  Coraluppi 

(stefano.coraluppi@stresearch.com). 

2)  Professor  Willsky  is  consulting  with  Dr.  Jeffrey  Byrne  (jeffrev.bvme@stresearch.com) 
on  exploiting  efficient  methods  of  large-scale  inference  for  Gaussian  graphical  models 
(one  of  the  major  components  of  our  AFOSR  program)  for  problems  of  visual  object 
recognition  under  a  DARPA  program. 

3)  Our  new  methods  for  dynamic  nonparametric  models  employing  hierarchical  Dirichlet 

processes  are  being  used  at  STR  for  the  characterization  of  motion  behavior  patterns 
under  several  programs.  The  STR  points  of  contact  are  Dr.  Peter  Jones 
(peter,  j  ones@stresearch.  com)  and  Dr.  Kirill  Trapeznikov 

(kirill.trapeznikov@stresearch.com). 

4)  Prof.  Willsky  has  worked  with  engineers  at  PSl  on  transition  of  message-passing 
algorithms  for  multi-object  tracking,  and  our  methods  for  nonparametric  learning  of 
dynamic  behavior  was  utilized  in  several  AF  programs  (now  completed)  at  PSl.  The  PSl 
point  of  contact  is  Dr.  John  Fox  (john.fox@parietal-svstems.com). 
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